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y Delayed  Delta  Modulation  (DDM)  uses  a  second  feedback  loop  in  addition  to 
the  standard  DM  loop.  While  the  standard  loop  compares  the  current  predictive 
estimate  of  the  input  to  the  current  sample,  the  new  loop  compares  it  to  the 
upcoming  sample  so  as  to  detect  and  anticipate  slope  overloading.  Since  this 
future  sample  must  be  available  before  the  present  output  is  determined  and  the 
estimate  updated,  delay  is  introduced  at  the  encoding. 

The  performance  of  DDM  with  perfect  integration  and  step-function  reconstruc¬ 
tion  is  analyzed  for  each  of  three  inputs.  In  every  case,  the  stochastic  stability 
of  the  system  is  established.  For  a  discrete  time  i.i.d.  input,  the  (limiting) 
joint  distribution  of  input  and  output  is  derived,  and  the  (asymptotic)  mean  square 
sample  point  error  MSE(SP)  is  computed  when  the  input  is  Gaussian.  For  a  Wiener 
input,  the  joint  distribution  of  the  sample  point  and  predictive  errors  is  derived, 
and  MSE(SP)  and  the  time-averaged  MSE  (MSE(TA))  are  computed.  For  a  stationary, 
first-order  Gauss-Markov  input,  the  joint  distribution  of  input  and  output  is 
derived,  and  MSE(SP)  and  MSE(TA)  computed.  Graphs  of  the  MSE’s  illustrate  the 
improvement  attainable  by  using  DDM  instead  of  DM.  With  optimal  setting  of  param¬ 
eters,  MSE(SP)  (MSE(TA) )  is  reduced  about  15%  (35%).  , 
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I.  DELTA  MODULATION  AND  DELAYED  DELTA  MO^fAT?Wfriical  rnfn, 

°r0atlonDlvlsIon 

Feedback/quantization  schemes,  such  as  Pulse-Code  Modulation  (PCM),  Differen¬ 
tial  PCM  (DPCM),  and  Adaptive  PCM  and  DPCM,  have  been  studied  by  a  number  of 


authors  (cf.  [1] , [3] , [5-10] ,  [14]  and  [15]). 


These  schemes  all  have  the  same 


goal:  Instead  of  transmitting  the  entire  analog  signal  X^,  t£0,  they  use  only 
the  samples  X^,  n=0,l,2,...,  to  generate  and  transmit  digitally  an  appropriately 
coded  (e.g.  quantized  and  possibly  block  coded)  sequence  YnT,  n=0,l,2,...,  from 
which  an  analog  output  Y  ,  t^O,  is  reconstructed,  which  should  approximate  the 
input  as  closely  as  possible.  The  analog  input,  which  we  denote  by  X^  through¬ 
out,  is  usually  taken  to  be  some  random  process.  The  sampling  interval  T>0  is 
fixed,  and  we  employ  (generically)  the  notation  Xn=XnT  throughout  as  well. 

Delta  Modulation  (DM)  is  the  simplest  of  the  differential  feedback/quantiza¬ 


tion  schemes.  Using  either  perfect  (p=l)  or  leaky  (0£p<l)  integration,  the 
DM  y“  of  Xt  is  generated  as  follows: 


yDM 

n+1 


n=0,l ,2, . . . 


where  the  step  size  A>0  and  bjj^  =  sign(Xn+1~  pYR)  .  For  te[nT,  (n+l)T) ,  the  value 
of  Y^1  is  appropriately  defined,  depending  on  the  system  and  the  input,  either 
in  terms  of  the  present  and  earlier  values  of  Yn  (predictive  reconstruction)  or  in 
terms  of  neighboring  past  and  future  values  of  Y^  (interpolative  reconstruction). 
When  an  interpolative  reconstruction  in  used,  the  reconstruction  is  delayed  until 
the  required  future  value(s)  of  Yn  are  available.  Frequently,  the  "step-function" 
(predictive)  reconstruction  Y  =  Y^  for  tc[nT, (n+l)T) ,  n=0,l,2,...  is  considered. 
More  sophisticated  reconstructions  will,  of  course,  yield  better  performance. 

It  is  well  known  that  the  size  of  A  determines  the  trade-off  between  the 
granular,  or  round-off,  error  of  the  system  and  the  slope  overload  error.  In  the 


.2 


DM  (with  perfect  integration  and  step- function  reconstruction)  realization  of 
Figure  1,  [8T,12T]  is  an  interval  of  slope  overload.  Note  how  the  DM  response 
at  time  7T,  though  locally  optimal,  exacerbates  the  effect  of  the  slope  over¬ 
load  suffered  immediately  afterwards.  Clearly,  an  upward  step  would  have  been 
preferable  to  the  downward  step  taken,  but,  a  priori,  there  was  no  way  of  knowing 
this.  If  we  knew  when  a  slope  overload  was  to  occur  we  could  reduce  its  effect 
by  anticipating  it  and  getting  a  "head  start"  on  it.  This  suggests  the  following 


scheme,  for  c£0. 


Y-1  *° 


0  if  |x^|  £  cA 

A  sign(X^)  if  |X^|  >  cA 


and  for  n=0 ,1,2,..., 


where 


Y„*i.  <*„  *“w 

b  __  jsign(X^rpYn)  if  Ix^jl  £  ci  , 
ntl  '  IsignCX^j)  if|X'+1|>ca. 


Unfortunately,  this  scheme  requires  sampling  both  the  signal  Xt  and  its  deriva¬ 
tive  X^.  The  following  scheme  is  similarly  motivated  but  requires  sampling  X^ 


only,  for  c^O, 


Y-i  ■  °- 


and  for  n=0 ,1,2,., 


where 


if  | Xx |  s  cA  , 


A  sign(Xj)  if  |Xj|  >  cA  , 


PY„  ♦  Ab  . , 


signtX^j-pV  if  |Xn,2-  Yj  £  ci  , 
signCXn<2-pY_)  if  |X_.,-  Y_|  >  ca  . 


1.3 


We  call  this  Delayed  Delta  Modulation  (DDM)  because  is  not  determined 
until  Xn+1  has  been  sampled.  Thus,  delay  ir  introduced  at  the  encoding  step 
(and,  if  we  wish,  at  the  decoding  step,  i.e.  the  reconstruction,  as  well). 

To  facilitate  the  analysis  and  allow  comparison  with  the  results  of  Fine  [5], 
Slepian  [15]  and  Masry  and  Cambanis  [14]  we  henceforth  consider  only  perfectly 
integrated  DDM,  and  so  take  p=l,  with  the  step  function  reconstruction, 

Yt  =  Y^  for  te  [nT,  (n+l)T) ,  n=0,l,2,...  .  Figure  1  includes  both  the  DM  and 

(c=l)-DDM  outputs  of  the  given  input. 

Note  that  setting  c  =  00  reduces  DDM  to  DM.  This  enables  us  to  check  our 
computations  against  the  literature.  When  we  set  c  =  0,  the  DDM  output  is  just 
the  DM  output  shifted  to  the  left  one  sampling  interval  T,  i.e.,  =  Y^^  . 

A  system  using  greater  delay  and  additional  comparisons  would  undoubtedly 
outperform  this  basic  DDM  scheme.  Unfortunately,  adding  non-linear  feedback 
loops  to  non-linear  differential  quantization  schemes  greatly  increases  the 
difficulty  of  their  analysis. 


II.  STOCHASTIC  STABILITY  OF  DDM 


Several  types  of  "stochastic  stability"  have  been  considered  for  feedback/ 
quantization  of  random  inputs.  Henceforth,  we  let  e^  *  X^-Y^  denote  the  sample 
point  error  (at  instant  n) . 

Gersho  [ 8  ]  (Hayashi  [ 9  ])  has  shown,  for  DM  (with  perfect  integration)  of 
a  stationary  and  continuously  distributed  input  that  has  finite  variance  and  a 
rational  spectral  density  (of  an  input  that  has  i.i.d.,  zero-mean,  finite  vari¬ 
ance  increments),  that 

(i)  the  distribution  of  (X^ , Y^)  (of  e^)  converges  weakly  to  a  unique  sta¬ 
tionary  steady-state  distribution  irrespective  of  the  initial  (output)  state, 
and 

(ii)  n  ^le]J  *s  un^^or,nly  bounded,  and  E | e |  <  °°,  where  e  is  the  limit 

in  distribution  of  e  . 

n 

Kieffer  [12]  considers  several  other  types  of  stochastic  stability.  Letting 
X~  =  (Xk,XJt+1, .. .) ,  he  shows  that  for  DM  of  a  stationary  input,  n"1^^  ^Xk'Yk^ 
converges  almost  surely  as  n  +  “  for  every  bounded  measurable  f . 

A  stronger,  and  more  useful,  notion  of  stability  is  what  may  be  called  "rth 
moment  stochastic  stability."  We  say  an  analog-digital -analog  system  is  rth 
moment  stochastically  stable  when  for  an  input  that  is  either  stationary  with 
finite  rth  moment  or  has  i.i.d.  zero-mean  finite- rth-moment  increments 

(a)  en  =>e,  whose  distribution  does  not  depend  on  the  initial  state  and  is 
stationary  for  the  sequence,  and 

,n-l  t  T 

(b)  lim  n"1  £  |e.  |  =  E | e |  <  <*>  a.s. 

k=0  K 

It  is  well  known  that  DM,  and  DPCM,  with  leaky  integration  satisfy  (a)  and  gene¬ 
rate  a  uniformly  bounded  output.  Thus,  for  a  stationary  input  they  satisfy  (b) 
trivially  and  so  are  r^  moment  stochastically  stable.  For  systems  with  perfect 
integrators  (b)  is  far  more  difficult  to  obtain  and  may  be  replaced  by 


II. 2 


(b’) 

lim  Ej 

|enr  s  B|e|r  <  »  , 

n+°° 

,n-l 

(b") 

lim  n" 

l  Eje  |r  =  E|e|r 

ir*°° 

k=0 

ox  even  for  some  v<r, 

(b,M)  lim  n"1  J  E | e  | v  =  E| e|V  <  <»  . 
n-w®  k=0 

The  averagings  used  by  Gersho  [ 8  ]  and  Hayashi  [ 9  ]  can  be  easily  extended  to 

show  that  DM,  and  DPCM,  with  perfect  integration  satisfy  (b'M)  for  v=r-l.  In 

Theorem  1  below  we  present  a  uniform  bound  on  |yDDM  _  yDM|.  Hence,  DDM  with 

n  n  1 

leaky  (perfect)  integration  satisfies  (b)  ((b’")  with  v=r-l),  and  so  is  rth 

moment  (v=(r-l)th  moment)  stochastically  stable  whenever  (a)  holds. 

Henceforth,  when  we  say  simply  that  DDM  is  stochastically  stable  we  mean 

only  that  (a)  is  satisfied.  In  the  following  subsections  we  will  show  that  DDM 

with  prefect  integration  is  stochastically  stable.  The  modifications  needed  to 

show  that  the  same  is  true  for  the  leaky  integrator  are  straightforward. 

In  order  to  establish  the  results  quoted  earlier,  Gersho  [ 8  ]  embeds  both 

the  input  X  and  output  Y  in  a  "state  vector"  S  ,  while  Hayashi  [9  ]  considers 
n  n  n 

’  WV  the  predictive  error.  They  show  that  (e*)  is  Markovian,  and  that 
n  EIY]J  C  Ele£l  ^  *s  unifonnly  bounded.  Thus,  the  sequence  of  ave¬ 

raged  distributions  of  Sn  (e*)  is  tight,  and  so  Sn  (e*)  has  a  stationary  distribu¬ 
tion.  By  verifying  certain  conditions  on  the  transition  function  of  S  (e*) ,  it 

n  n 

-follows  that  the  distribution  of  (e*)  converges  weakly  to  a  unique  (stationary) 
steady-state  distribution  irrepsective  of  the  initial  state  SQ  (e*)  .  Our  approach 
parallels  theirs. 

The  following  uniform  bound  is  very  useful.  From  it  we  obtain  tightness  and 
finiteness  of  moments  for  DDM  as  a  consequence  of  the  same  for  DM.  This  bound 
can  be  shown  to  be  tight  for  cj*0,  but  the  argument  will  not  be  produced  here. 


II. 3 


Theorem  1  For  any  given  input,  perfectly  integrating  systems  (p=l),  and  c£0,  we 
have,  for  all  n, 

|YDDM  _  yDM|  s  (c+3)A  _ 

Proof:  Let  Dn  =  y^DDM^  -  Y^M,  n20.  Clearly  (i)  DQ  *  0,  (ii)  Dj  =  -A,0,  or  A, 

and  (iii)  D  =  D  -  2A,  D  ,  or  D  +2A,  nsi. 

v  n+1  n  ’  n’  n  * 

The  bound  will  be  established  by  contradiction.  Let  nQ  <  «  be  the  smallest 


value  of  n  s.t. 


| Dn J  >  (3+c)A.  Then  |Dn  |  =  (dQ+2)A,  where  the  integer  dQ  satis¬ 


fies  d0£(l+c,  3+c].  Without  loss  of  generality,  assume 


%  ■  <V2>4  • 


(2.1) 


Let  nj  denote  the  largest  value  of  n  less  than  nQ  such  that  =  (dQ-2)A. 


Then 


Dni  =  (d0-2)A  and  -  dQA  . 


(2.2) 


and  so  b  =1  and  bDM  *  -1.  This  yields' {(X  ->Y  +cA)  u  (|x  _-Y  |  <  cA 

n1+l  n^  n1  nj  +  ^  nj 

and  X  .>Y  )}  n  {X  .SY^M),  and  since  Y  *  Y1^  +  (d  -2)A  t  YDM,  this  reduces 

nj+1  nj  n1+l  n^  nj  v  0  n^* 


to  X 


V2 


>  Y  +  cA  and  X 


n. 


V1 


s  Y 


DM 


From  the  first  of  these  inequalities  and 


(2.2)  we  derive 


XV2  »  *  <V2>  *  CA 


>  YDM  +  (l+c-2) A  +  cA 
nl 


i.  YDM  -  A  +  2cA 


a  YDM  ♦  Ab™  ♦  2cA 

ni  V1 


2  Y 


DM 

nl+l  ' 


Hence 


b  _  *  1.  But  by  (2.1),  D  -  =  dnA.  Thus  b  ,  *  1  as  well 
n, +2  n,+2  o  n, +2 


This  implies  that 


X  >  Y  +  d.A  -  cA 

nj+3  nj+1  0 


>  Y  +  (l+c)A  -  cA 


*  ydm.,  *  A 

V* 


I  M  • 

nj+2 


Hence  b 


>  =1.  But  by  (2.1),  D 

nx  +  3  ^+3 


=  d„A.  Thus  b  =  1  as  well, 


Repeating  this  series  of  steps  (n.-n. )  times  yields  D  =  D  A,  in  contra- 

"  n0  0 

diction  of  (2.1) .  □ 


A.  Stationary  input 


We  show  first  that  DDM  is  stochastically  stable  for  an  input  Xt,  t£0,  that 
is  either 

(i)  stationary  Gaussian  with  rational  spectral  density,  or 
(ii)  stationary  Markovian  with  finite  variance,  uniformly  bounded  and  every¬ 
where  positive  multivariate  densities  of  all  orders,  and  uniformly  boun¬ 
ded  conditional  densities  of  Xn+1  given  Xn,...,  XQ. 

Processes  of  the  former  type  satisfy  all  but  the  Markov  requirement  of  the  latter 
type.  However,  as  noted  by  Slepian  [153,  they  can  be  embedded  as  the  first  com- 
ponent  in  a  stationary  Gauss-Markov  row-vector  process  Xt,  t^O,  that  is  of  the 
latter  type.  Thus,  with  a  slight  abuse  of  notation,  we  take  Xr  to  be  a  (possibly 
one-dimensional)  row-vector  that  satisfies  the  conditions  in  (ii)  and  we  put 

Sn  *  (Y_,  X_,  X_.J,  n=0,l,2, . . .  .  (2. 


1.  {S  }  is  Markovian 
— n - 

From  the  definition  of  DDM  we  see  that 


y„.i  ■  *'V  Vi-  X„.2>  -  e'V  • 

and  so 

Yn  =  hn(Xn+l”**’  V  * 

Hence,  with  probability  one, 

S  , |(S  ,. 
n+1 1 v  n* 


(2.4) 


V 

% 

fVi-VrWl 

l(Yn.- 

n 

IV  IV 

& 

(Yn+l’Xn+2*Xn+3) 1 

l(Yn’  : 

Oil 

(Yn+l’*n+2’*n+3)  1 

O’  n+2’ 


=  S  ,  S  , 
n+1 1  n 


and  so  Sn  is  a  Markov  process. 

2.  (Sn)  has  a  stationary  distribution  G 

Gersho  [8  ]  has  shown  that  the  averaged  distributions  of  Y^M  are  tight,  and, 
as  a  consequence  of  Theorem  1,  the  same  holds  for  Y^DJ .  By  stationarity,  it  is  also 

IV 

true  of  Xn<  Thus  the  averaged  distributions  of  the  components  of  are  tight, 
and  hence  so  are  the  averaged  distributions  of  Sn> 

Let  Fn  denote  the  distribution  of  Sn,  and  Gfi  =  Fk’  Comttt: ioning 

backwards  gives  us 

W«  ’  /  P<Sn.l£AlS„-s>dFn's>  • 

From  (2.3),  (2.4),  and  the  fact  that  ^  is  Markovian  it  follows  that  the  transition 

function  P(S  ,eA|S  =s) does  not  depend  on  n.  Hence  we  may  write  F  =  TF  . 

n+1  n  n+x  n 

For  a  set  A,  let  DA  denote  the  set  of  points  s  such  that  P(Sn+1eA |Sn=s)  is 

6 

discontinuous,  and  let  denote  the  set  of  points  within  6  of  D^.  The  following 
result  is  Gersho’s  [8],  but  the  proof  has  been  simplified. 


II. 6 


Theorem  2.  Let  Sn  be  a  Markov  process  with  averaged  distributions  Gn  and  tran¬ 
sition  function  P(Sn+jeA|Sn=s)  that  does  not  depend  on  n.  If 

(1)  the  sequence  Gn  is  tight,  and 

(2)  for  any  open  set  A,  there  exists  a  function  c^(<5)  ■+•  0  as  6  -*■  0  such 
that  P(SneD^)  s  cA(6)  for  all  n, 

then  SR  has  a  stationary  distribution. 

Proof :  Because  (Gn)  is  tight,  there  exist  a  subsequence  G^  and  a  distribution  G 

such  that  G  =>  G.  But  TG  =  G  +  n_1(F  -F.)  and  so  TG  G. 
n^  n  n  'no  n.  i 

From  (2)  we  have  Fn(DA)  £  cA(6)  for  all  n,  and  so  Gn(DA)  s  cA(6) .  Since  DA 

X 

is  open,  G(DA)  s  cA(6)  so  that  G(DA)  =  0  for  all  A  open. 

Theorem  5.2.iii  on  page  31  in  Billingsley  [2]  now  gives  us  TG  (A)  -*■  TG(A)  for 

ni  i 

D  1 

all  A  open,  and  therefore  TGn  =>TG.  Hence  TG  =  G.  □ 

ni  1 

In  order  to  apply  this  theorem,  it  remains  only  to  show  that  the  second  con¬ 
dition  is  satisfied.  The  following  lemmas  prove  that  we  may  take  cA(6)  =  c(6)  =8M6. 

m 

x. 

Lemma  1.  For  DDM,  for  any  Borel  set  A,  5<A,  and  all  n,  we  have  P(SncDA)  £  8MS, 
where  M  is  a  uniform  bound  on  the  conditional  density  of  Xn+^  given  X^, . . . ,  XQ. 

oo  A 

Proof:  Observe  that  for  DDM,  for  any  region  A,  DA  c  D  =  IL_  w  D^,  where 
0^^  =  {(slfs2,s3):  Sj  *  iA,  and  (s^  «  iA  *  cA  or  s^s  iA)},  and  thus 
DA  c  =  U”.^  D?.  Since  for  <5<A, 

P(SneD^)  =  P(Yn  *  iA,  |Xn+2-iA-cA|<6)  +  P(Yn  =  iA,  |Xn+2-iA*cA|<6) 


♦  P(Yn=iA,  |Xn+1-iA|<6)  , 


while 


P(Yn-iA,  |Xn+1-iA|<5)  =  P(Ynl«(i-l)A,  bn  «  1.  |Xn+1-iA|<6) 


♦  P(Yn_1*(i+l)A,  bn  -  -1,  |Xn+1-iA|<6) 

S  P(Yn-l*(il)A'  lXn+riA|<6)  +  P(Yn-r£i+1)A'  lXn-riA,<6) 
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we  see  that  foT  6<A, 

00 

P(SneD^)  s  ^  {P(Yn=iA,  |Xn+2-iA-cA|<6)  4  <>(Yn=iA,  |Xn+2-iA+cA|<6)  ^ 

+  PCYn-r(i“1)A*  lXn+riA|<6)  +  P(Yn-l  =  (i+1)A*  lxn+riA|<6)}’ 

But  for  any  k,K,  and  m, 

P(Vki>  lXm.2-K  l<6>  ’  ’’tlW'l  *  «|Vn=kA)PCYm-U) 
s  26M  •  P(Ym=kA)  (2.f 

by  (2.4),  Lemma  2  below,  and  assumption.  Hence,  combining  (2.5)  and  (2.6)  we  ob¬ 
tain  our  result.  □ 

In  the  following  lemma,  U  and  V  are  i-  and  j-dimensional  random  vectors, 
respectively,  and  A  is  i-dimensional  Lebesgue  measure. 

Lemma  2.  Suppose  fy|v(u|v)  s  M  <  ®  a.e.  Then  for  all  Borel  AeF1  and  BelR^  , 
P(UeA| VeB)  <  MA (A) . 

Proof:  P(UeA,VeB)  =  JJ  f y | v (u | v) f y ( v) dvdu 

AB 

s  JJ  Mf  (v)dvdu  =  MA(A) P(VeB) .  □ 

AB 

3.  {Sn}  "converges’1  in  distribution 

Doob  [  ;  Theorem  5]  gives,  for  a  Markov  process,  conditions  sufficient  for 

a  stationary  distribution  to  be  the  (periodic  or  aperiodic)  steady-state  distri¬ 
bution.  It  is  a  simple  matter  to  verify  that  these  conditions  are  satisfied. 

(cf .  Gersho  [  8 ]  and  Gerr  [  7 ]) .  Since  the  sequence  (A  ^Yn)  alternates  between 
odd  and  even  values,  the  period  two  result  obtains  (cf.  Gerr  [  7  )  for  more  de¬ 
tails)  . 

B,  Iti,d.  increments  input 

When  the  input  to  a  DDM  system  has  i.i.d.  increments,  such  as  a  Wiener  pro¬ 
cess,  the  joint  distribution  of  input  and  output  clearly  cannot  converge. 


II. 8 


However,  when  the  perfect  integrator  is  used  the  predictive  error  e*  =  xn+1"Yn 
does  converge  in  law. 

We  let  I  ,  =  X  A.-X  ,  n=0,l,...  and  assume  that  the  I  's  form  an  i.i.d.  se- 
n+1  n+1  n  n 

quence  with  mean  zero,  variance  T,  and  bounded,  everywhere  positive  density  f ^ . 
These  assumptions  are  satisfied  by  the  standard  Wiener  process. 

1 .  {e*}  is  Markovian. 


It  is  easy  to  see  that 


e*  =  I  _  +  e*  -  Ab  ,  , 
n+1  n+2  n  n+1  ’ 


(2.7) 


where 


b  ,  =  sign(e*  +  I  A->)1  ,  .  .  (e*  +  I  _)  +  sign(e*)l,  .  .  .,fe*  +  I  _) 

n+1  B  n  n+2'  ±(cA,°°)^  n  n+2-*  s  L  n'  [-cA.cA]'"  n  n+2-' 


and  so 


=  •  W  * 


en+l  *  g(en*  Tn+2} 


(2.8) 


en  =  hnfIn+l'”"  V  ' 


(2.9) 


Since  the  I  's  are  i.i.d.,  it  follows  that  e*  is  Markovian, 
n  n 

2.  {e*}  has  a  stationary  distribution. 

Hayashi  [ 9  ]  has  shown  that  for  DM  of  Wiener  process  input,  the  sequence  of 

averaged  distributions  of  e*DM  is  tight  for  all  A  >  0.  It  is  easily  seen  that 

this  result  can  be  extended  to  include  any  process  that  has  zero-mean,  finite 

variance,  i.i.d.  increments.  Since  |e*-e*^|  =  |YbbM-Yb^|  <  (c+3)A  by  Theorem 

n  n  1  n  n  1 

1,  the  sequence  of  averaged  distributions  of  e*  is  tight  for  all  A  >  0. 


Letting  denote  the  distribution  of  e*,  we  again  have  that 


f„.i«  ■  /  Ft*;., « *i«:  -  >)«„(•)  . 


From  (2.7-9)  we  see  that  the  transition  function  P(e*+1e  Aje*  =  s)  does  not  depend 

on  n,  and  so  we  may  write  F  .  =  TF  .  It  follows  from  (2.9)  that  e*  and  I  are 

n+l  n  n  n+2 

independent.  Since  bn+j  =  ±1,  from  (2.7)  we  see  that  we  may  take  c^(6)  =  c(6)  = 

2M6,  where  M  =  sup  fT.  Thus  both  conditions  of  Theorem  2  are  satisfied  and  e* 

1  n 

has  a  stationary  distribution  G. 

3.  {e*}  converges  in  distribution. 

Once  again  the  conditions  of  Doob's  Theorem  are  easily  verified  (cf.  Gerr 
C  7  ]),  but  in  this  case  the  aperiodic  result  obtains. 

C.  I.i.d.  input 

We  show  here  that  if  the  input  Xn  is  a  discrete  time  i.i.d.  process  that  has 

bounded  everywhere  positive  density,  the  joint  distribution  of  input  and  output 

A 

converges  weakly.  Let  S  =  (Y  ,  X  , ,  X  _) . 

n  n’  n+l’  n+2' 


1.  {S  }  is  Markovian. 

— n - 

This  follows  from  Subsection  A.l,  as  Xfi  is  both  stationary  and  Markov. 

2.  {Sn>  has  a  stationary  distribution  G. 

Because  Xn  is  stationary.  Subsection  A. 2  implies  that  the  averaged  distribu¬ 
tions  of  S^  are  tight.  As  the  Xn  are  independent  with  bounded  density  the  second  con 
dition  of  Theorem  2  is  also  satisfied.  Hence  S^  has  a  stationary  distribution  G. 

3.  {S^}  "converges"  in  distribution. 

Once  again,  the  conditions  of  Doob’s  Theorem  are  easily  verified,  and  in  this 
case  the  period  two  result  obtains. 


III.  DDM  OF  DISCRETE  TIME  I.I.D.  INPUT 


To  analyze  the  response  of  DDM  to  a  discrete  time  i.i.d.  imput  Xn>  n=0,l,2,.. 
we  adapt  the  approach  taken  by  Fine  [ 5  ]  in  his  study  of  DM  of  this  input.  First 
we  find  the  stationary  joint  distribution  of  input  and  output.  This  is  done  by 
solving  the  Chapman-Kolmogorov  Equation  (CKE) .  Then,  assuming  that  the  input  has 
a  standard  normal  distribution,  we  compute  the  asymptotic  sample  point  mean 
squared  error  (MSE(SP))  as  a  function  of  c  for  several  values  of  A  and  as  a  func¬ 
tion  of  A  for  several  values  of  c. 

Recall  from  Section  II. C  that  when  we  write 

F. (x)  =  lim  P(Y  =  iA,  X  <x) 

1 v  n  n 

n 

we  in  fact  mean 

F.(x)  =  lim  %{P(Yn  =  iA,  Xn<x)  +  P(Yn+J  =  iA,  Xn+1<x)}  . 
n 

In  the  following,  all  expressions  will  be  written  using  the  simplified,  rather 
than  the  precise  averaged,  form. 

The  input  Xn>  n=0,l,2,...,  is  assumed  to  be  i.i.d.  symmetric,  with  finite 
second  moment  and  common  distribution  H  that  has  density  h. 

A.  Analysis. 

1.  The  asymptotic  distribution  of  (Yn>  Xn[1). 

Let  F?(x)  =  limn  P(Yn = iA,  xn+1Sx) .  i=0»*l»  *2,...  .  The  existence  of  these 
weak  limits  was  shown  in  II. C.  In  (3.1)  we  show  that  these  distributions  have 
densities  and  may  be  written  in  terms  of  the  input  distribution  and  the  constants 
pA  =  limn  P(Yn  =  iA) ,  and  q^  *  F|(iA),  for  which  we  derive  an  infinite  system  of 
homogeneous  linear  equations  (3.2-5)  that  is  solved  by  the  Method  of  Reduction. 

By  conditioning  backwards  we  obtain  a  CKE  for  Ft: 


F I«  -  =  (i‘1)A*  bn  =  Xn+l“x^  +  P<Yn-l  =  ^i+1)A'  bn  =  -1-  Wx) : 3 

=  Pi.1CH(x)-H((i-l+c)A)]+  +  (pi_1-qil)[H(min(x,(i-l+c)A))  -  H( (i-l-c)A) ]+ 

+  Pi+jH(min(x, (i+l-c)A))  + q.+1[H(min(x, (i+l+c)A))  -  H((i+l-c)A) ]+  (3.1 

where  [a]+  =  a  if  a£0,  [a]+  =  0  if  a<0. 

It  follows  from  (3.1)  that  Ft  has  density  ft,  and  so  the  endpoints  neglected 

in  the  derivation  are  of  no  concern.  Specifically,  we  can  write  ft(x)  =at(x)h(x), 

where  h  is  the  density  of  H  and  the  step  function  at  is  easily  identified  by  (3.1) 

(see  Gerr  [  ]) .  It  is  also  clear  that  the  constants  {p.,q.}°°_  determine  the 

distributions  (Ft(*)}.  „  . 

1  1=  -  00 

By  letting  x  -*•  00  in  (3.1)  we  obtain 
Pi  =  Pi_1Cl-H((i-l-c)A)]  -  qi.1^H((i-l+c)A)-H((i-l-c)A) ] 

♦  Pi+1H((i+l-c)A)  +  q.+1[H((i+l+c)A)-H((i+l-c)A)]  ,  (3.1 

and  taking  x  *  iA  yields 

qi  =  pt  1[H(iA)-H((i-l+c)A) ]+  +  (pil-qil)[H(min(iA,(i-l+c)A))-H((i-l-c)A) ]+ 

+  Pi+1H(min(iA, (i+l-c)A))  +  qi+1(H(min(iA, (i+l+c)A))-H((i+l-c)A) ]+  .  (3.2 

The  recursions  (3.2  and  3)  define  an  infinite  system  of  homogeneous  linear 

r  i00 

equations  in  „  •  In  addition,  since  the  DDM  system  and  input  are  sym¬ 

metric,  the  output  is  symmetric,  i.e. 

P_i  =  Pi  »  (3-4 

and  since  the  input  has  density, 

q  .  •  lim  P(Y  =  -iA,  X  .  £  -iA)  =  lim  P(Y  =  iA,  X  .  2  iA)  =  p.  -  q..  (3.5 

^-i  v  n  *  n+1  n  n+1  '  *i 

n  n 

The  infinite  homogeneous  system  given  by  (?.2-5)  is  supplemented  by  the  fol¬ 
lowing  non-homogeneous  normalization  requirements.  Due  to  the  periodicity  of 
the  output  and  the  limiting  stationary  distribution 


III. 3 


£  p  =  Jj  a  l  ?i  ,  (3.6) 

i  even  i  odd 

and  so  from  (3.5), 

I  qA  -  %  -  I  qA  .  (3.6.i) 

i  even  i  odd 

The  system  given  by  (5.2-6)  is  solved  by  the  Method  of  Reduction  (see  the 
Appendix),  and  we  recover  the  distribution  Ft  using  (3.1). 

2.  The  asymptotic  (mean  square)  sample  point  error. 

As  noted  by  Fine  [  ], 

F  (x)  =  lira  P(e  <x)  =  lira  P(X  -Ysx) 
e  n  nn 

n  n  (3.7) 

=  lim  l  P(Yn  =  iA,  X^sx+iA)  =  £  F.(x+iA)  , 
n  i  i 

where  F.(x)  =  lim  P(Y  =  iA,  X  Sx) .  The  distributions  {F.(*)}.  are  derived 
in  terms  of  the  distributions  (Ft(»)}^_  m  by 

F.(x)  -  lim[P(Yn_1  =  (i-l)A,  bn  =  1,  X^x)  +  PfY^  =  (i*l)A,  bn  -  -1,  Xn<x)  ] 
n 

-  F*  1(x)[l-H(i-l+c)A)]  +  CF*_1(x)-qi-1D+  [H( (i-l+c)A)  -  H((i-l-c)A)] 

+  Fr+1(x)H((i+l-c)A)  +  F^+1(min(x, (i+l)A))[H((i-l+c)A)  -  H((i-l-c)A)]  .  (3.8) 

Since  Ft  has  density  ft,  it  follows  from  (3,8)  that  F^  has  density  f^,  which  for 
c>l  (which  turns  out  to  be  the  region  of  greatest  interest)  is  given  by 

fi(x)  =  ft_1(x)H((l-i-c)A)  +  f?+1 (x)H( (i+l+c) A) ,  x<(i-l)A, 

ft_1(x)H((l-i+c)A)  *  ft+1(x)H((i+l+c)A) ,  (i-l)A  s  x<(i+l)A 

f*_1(x)H((i-l+c)A)  +  ft+1(x)H((i+l-c)A),  (i+l)A<x. 

Substituting  the  expressions  for  ft  we  find  f^(x)=a^(x)h(x) ,  where  the  step 
functions  a^(x)  are  given  in  Gerr  [73. 

Having  found  f^  it  is  straightforward  to  derive  the  asymptotic  sample  point 
mean  square  error.  Letting  e  denote  the  limit  in  distribution  of  e^  =  X^-Y^, 
from  (3.7)  we  derive 


Ill  .4 


MSE(SP)  =  X2f  (x)dx  =  l  x2f  (x+iA)dx  =  l  (x-iA)2f  (X)dx 

i  i 

=  l-2^  l  i  /Too  xf.(x)dx  ♦  A2!  i  2Vi  •  (3.9) 

B.  Numerical  results  and  discussion. 

Recall  that  the  infinite  system  (3.2-6)  is  solved  by  the  Method  of  Reduction. 
This  involves  setting  p^Osq^^  for  |i|  >  I,  solving  the  resulting  finite  system 
exactly,  and  taking  as  the  solution  for  the  infinite  system  the  limit  as  I-*»  of 
these  "approximating  solutions."  The  Theory  of  Majorants,  developed  by 
Kantorovich  and  Krylov  [11],  and  presented  in  the  Appendix,  assures  us  that  the 
Method  of  Reduction  results  in  solution  of  the  system  encountered  here. 

We  calculate  MSE(SP)  when  the  i.i.d.  input  is  standard  normal.  Regarding 
the  Method  of  Reduction,  for  all  A  and  c,  the  value  of  MSE(SP)  computed  when 
1=4  agreed  with  the  1=5  value  to  about  four  significant  figures.  In  Table  I 
we  exhibit  the  output  level  probabilities  p^,  i=0,...,5  for  DDM  and  DM  with  op¬ 
timal  parameter  settings.  Note  that  these  probabilities  decrease  rapidly, 
facilitating  the  numerical  technique.  In  addition,  our  computations  for  DDM  with 
large  c  (i.e.  DM)  agree  with  Fine's  [5  3  for  DM  to  nearly  four  significant  figures. 

In  Figure  2  we  plot  MSE(SP)  as  a  function  of  c  for  A  =  .6,  .75  and  .9.  Ad¬ 
ditional  computations  have  shown  that  (A,c)Qpt  =  (.745,  2.29)  is  the  pair  of 
values  at  which  MSE(SP)  is  minimized  and  equals  .5691.  Fine  [5  ]  found  that  for 
DM,  AQpt  =  .77,  giving  MSE(SP)  =  .6402.  Thus,  the  reduction  in  MSE(SP)  of  opti¬ 
mal  DDM  versus  optimal  DM  is  about  11.1%.  Note  that  the  change  in  the  value  of 
the  optimal  A  is  quite  small.  However,  recalling  how  the  size  of  A  determines 
the  trade-off  between  round-off  and  slope  overload,  it  is  not  surprising  that 
the  optimal  A  is  smaller  for  DDM  than  for  DM.  The  additional  DDM  loop  is  designed 
to  mitigate  the  effect  of  slope  overloads,  which  previously  could  only  be  done 
through  the  use  of  larger  A. 
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The  shape  of  the  curves  in  Figure  2  is  typical  of  the  response  of  DDM  for 
the  inputs  considered.  However,  for  i.i.d.  input,  the  performance  of  DDM  is 
uncharacteristically  poor  when  c  is  small.  This  may  be  rationalized  as  follows: 
When  c=0,  Yn  tracks  Xn+j,  but  MSE(SP)  is  measured  relative  to  Xn,  which  is  inde¬ 
pendent  of  *n+j.  We  expect  that  when  the  input  is  (more)  positively  correlated, 
smaller  settings  of  c  will  result  in  greater  reductions  in  MSE(SP) . 

Fine  [  5  ]  has  pointed  out  1-2/tt  »  .3634  is  a  lower  bound  for  MSE(SP)  of  a 
predictive  feedback/quantization  scheme  that  has  a  fixed  two-level  quantizer 
when  the  input  is  i.i.d.  standard  normal,  and  that  this  lower  bound  is  attained 

j< 

only  by  PCM  with  quantizer  output  levels  ±  (2/tt)  «  *  .798.  This  is  not  sur¬ 

prising;  PCM  is  non-differential,  and  so  may  have  independent  output  values, 
but  DM  is  differential,  hence  its  output  values  are  necessarily  correlated. 
However,  this  lower  bound  does  not  apply  to  systems  that  use  delay,  such  as  DDM, 
or  perform  "tree  searches"  (cf.  Chan  and  Anderson  [3  3),  i.e.  are  intexpolative. 

In  Figure  3  we  plot  MSE(SP)  as  a  function  of  A  for  c  =  1.5,  2.25,  3.,  and  8. 
(which  corresponds  to  c  =  °°,  i.e.  DM).  The  relative  lack  of  dependence  of 
MSE(SP)  on  A  for  DM  of  i.i.d.  standard  normal  input  was  noted  by  Fine  [5  ],  and 
we  see  that  a  similar  result  holds  for  DDM.  In  addition,  we  find  the  improvement 
of  DDM  with  near  optimal  c  (=2.25)  over  DM  to  be  almost  uniform  with  respect  to 
A  in  its  range  of  interest. 

^  2 

For  the  input  considered  here,  the  increments  I  =  X  -X  ,  have  E(I  j  =  2 
r  ’  n  n  n-1  v  n' 

2 

and  E(I  ,  I  )  =  E(-X  )  =  -1,  so  that  adjacent  increments  have  correlation 
v  n+1  n  v  n'  J 

-1/2.  When  adjacent  increments  are  more  positively  correlated,  slope  overloads 
should  be  more  prevalent  and  we  would  expect  even  greater  improvement  in  the  per¬ 
formance  of  DDM  relative  to  DM,  and  for  lower  settings  of  c  as  well. 


IV.  DDM  OF  WIENER  PROCESS  INPUT 


The  analysis  of  the  response  of  DDM  (with  perfect  integration)  to  Wiener 
process  input  parallels  that  of  Janardhanan  [10]  for  DPCM  with  matched  integra¬ 
tion  of  stationary  first-order  Gauss-Markov  input.  In  Subsection  A  the  limiting 
distribution  of  the  predictive  error  is  shown  to  satisfy  an  integral  equation  ob¬ 
tained  by  conditioning  on  the  preceding  step,  and  the  asymptotic  sample  point 
and  time-averaged  mean  square  errors  are  expressed  in  terms  of  this  limiting  dis¬ 
tribution.  In  Subsection  B  we  present  computational  results. 

The  input  Xt,  t^O,  is  the  standard  Wiener  process.  The  increments 
=  Xn-Xn  . ,  n=l,2,...,  are  i.i.d.  N(0,T),  and  we  let  <t>T  denote  their  density. 

A.  Analysis 

From  Subsection  II. B  we  have  that  the  distribution  F*(x)  =  P(e*<x)  has  density 
f*  and  converges  weakly  to  a  distribution  F*  having  density  f*.  f*  is  shown  to 
satisfy  an  (CKE)  integral  equation  (4.2),  which  is  solved  by  Galerkin's  Method 
(see  the  Appendix).  The  joint  density  of  e*  and  e,  the  limits  in  distribution  of 

e*  and  en  =  Xn-Yn,  respectively,  is  then  expressed  in  terms  of  f*  in  (4.3).  This 
yields  the  limiting  distribution  of  en(t)  =  xnT+t’Yn'  enabling 

the  calculation  of  the  asymptotic  predictive,  sample  point,  and  time-averaged 
mean- square- errors  (4.13,  14,  and  16). 

1.  The  density  f*  of  e*. 

Since  the  In's  are  independent,  from  (2.9)  we  have  that  e*  ^  and  In+j  are 
independent.  Thus,  using  (2.7)  we  derive 

*  p<e;-rx-£-w 

■  C  -  FJ  j(ci-u):* 

♦  [F*  , (min(x+A-u,cA-u))  -  F*  , (max(0,-cA-u)) ]+  +  F*  . (min(x-A-u.-cA-u)) 
n-l  n-i  n-i 

+  [Fn.i(minC°»cA-u,x-A-u))  -  F^  jt-cA-^j^j^^du  . 


(4.1) 


.2 


Since  F*  is  the  stationary  distribution  for  e*  it  satisfies  the  recursion 
(4.1).  Substitution,  differentiation,  and  a  change  of  variables  yields 

f*(x)  «  {[1 (x>(c-l)A)  +  l(-(C4l)A  <  x  <  (c-l)A)  /£)$t(x4A-v)  (4.2) 

4  [1 (x<-(c-l)A)  r.co  +  l(-(c-l)A  *  x  s  (c+l)A)  /°oo]<|)T(x-A-v)}f*(v)dv 


As  f*  is  a  density,  it  also  satisfies 
J  f*  *  1  and  f*  *  0  . 


(4. 2. 1,2) 


From  (4.2)  we  see  that  since  f*  is  integrable,  it  is  bounded,  and  hence 
square  integrable,  i.e.  f^I^Leb) .  In  Subsection  B  of  the  Appendix  we  show 
that  the  integral  operator  A  given  by  the  right  hand  side  of  (4.2)  is  bounded 
and  linear  on  I^Leb)  .  Thus  f*  is  an  eigenfunction  of  A  having  eigenvalue  one. 
We  now  show  that  the  manifold  of  integrable  solutions  to  f*Af  is  one  dimen¬ 
sional  . 

Theorem  3.  Suppose  f  is  integrable  and  satisfies  f=Af.  Then  f=rf*,  where 


Proof:  Examining  the  kernel  of  A,  we  see  that  since  f  is  integrable,  it  is 
bounded  and  continuous,  except  possibly  at  the  four  points  *  (c±l)A.  Let  f+  and 
f”  be  the  positive  and  negative  parts  of  f  and  set  p=/f+,  q=Jf'  and  r=Jf.  Then 
f=f+-f"  and  p-q=r.  In  Section  III.B  it  was  shown  that  TnFQ=F*=>F*  for  any  ini¬ 
tial  distribution  FQ.  Thus  for  all  bounded  continuous  functions  h,  JhAnf+ 4-pJhf* 
and  JhAf" -*■  q Jhf* ,  so  that  /hAnf  -*■  r/hf* .  But  Anf  =  An  *Af  =  An  *f  =  f  for  all  n. 
Hence  for  all  bounded  continuous  functions  h, 

Jhf  =  r/hf*  , 

and  since  f  and  f*  integrable  and  continuous  except  possibly  at  four  points, 
f  *  rf*  a.e.  .  0 

The  unique  integrable  solution  to  (4.2)  is  found  by  Galerkin’s  Method 


(see  the  Appendix)  . 


Let  Fn(u,v)  =  P(en^u,  e*<v) .  By  a  sequence  of  steps  similar  to  those  used 
at  the  beginning  of  the  previous  section  we  derive 

F  (u,v)  =  J00  {[F*  (min(u+A,v+A-w))  -  F*  . (cA-w)] + 

n  ~°°  n-i  n-x 

+  [F*_j (min(cA-w,u+A,v+A-w))  -  F*  ^ (max(0,-cA-w)) ]+ 

+  F*  j(min(-cA-w,u-A,v-A-w)) 

♦  [F*  ^ (min(0,cA-w,u-A,v-A-w))  +  F*  ^  (-cA-w)  ]+}<{>T(w)dw. 

From  (2.7  and  9)  and  II.B.3  we  have  that  F*  =>F*  that  has  density  f*.  As 
F*  is  uniformly  bounded  (by  one)  and  <f>^  is  integrable,  it  follows  that  con¬ 
verges  weakly  to  a  distribution  F  having  density  f.  The  joint  density  f  of  e 


and  e*  is  then  given  in  terms  of  the  density  f*  of  e*  by 

f(u,v)  =  (f *(u+A) [1 (v>(c-l)A) +1 (- (c+1) ASvS (c-l)A) 1 (u>-A) ] 


(4.3) 


+f*(u-A)  [1  (v<-  (c-l)A) +1 (-(c-1) A<v< (c+l)A)  1  (u<A) ]  }»<t>T(v-u)  . 


The  density  f(*)  of  e  is  the  marginal 

f(u)  =  /Too  f(u»v)dv. 


3.  The  sample  point  and  time-averaged  mean  square  error. 


Having  derived  the  joint  density  of  e  and  e*,  we  may  now  calculate  the  time 
averaged  mean-square-error 

MSE(TA)  =  y  jJ  E(e2(t))dt  , 

where  e(t)  is  the  weak  limit  of  en(t)  =  xnT+t"Yn" 

Let  Fn(x;t)  =  P(en(t)sx).  Then 

Fn(x;t)  «  //  P(en(t)sx|en  =  u,  e*  =  v)dFn(u,v)  . 


However , 


P(en(t)sx|en«u,e*«v) 


JP(en(t)Sx|e  *u,e*=v,Yn=iA)P(Yn*iA) , 


IV. 4 


-  P(XnT.t£x,ii|XnT*u*iA-X(n*l)T'v*iA)  • 
since  is  a  functional  of  X(n+i)j» • • • »  x0- 

Let  Bn(t)  =  XnT+t  -  [(1  -  ^)XnT  +  1  X(n+1)T]  for  te[0,T]  and  n=0,l,2,...  . 

Clearly,  for  all  n,  B  is  a  zero-mean  Gaussian  process  with  8^(0)  =  0  =  Bn(T) . 

Furthermore,  it  is  easily  shown  that  E(B  (s)B  ft))  =  sfl  -  4)  for  0<s<t<T.  Thus 

n  n  T 

for  all  n,  Bn(t),  te[0,T],  is  distributed  as  a  Brownian  Bridge  on  fO,T],  which 

we  will  denote  by  B_(t) .  In  addition,  since  E[B_(t)X  „]  =  0  =  E[B  (t)X,  -._) 

1  n  ni  n  (n+l j I 

for  all  n  and  te[0,T],  Bn(t)  is  independent  of  both  XnT  and  X(n+1)y  This  gives 
us 

P(XnT+t5x+iAlXnT=U+iA*X(n+l)T=V+iA) 

*  P(Bn(t)Sx+iA-t(l  -|)(u+iA)  +Ytv+iA)]|XnT=u+iA,X(n+1)T=v+iA) 

=  P(Bn(t)sx-[(l  -I)u+|-v])  =  P(BT(t)<x-[Cl -|)u+lv  ]),  (4.8) 

and  combining  (4.6-8)  that 

P(en(t)sx|en=u,e*=u)  =  P(BT(t)sx-[ (1  -  ^)u  +  |v])  .  (4.9) 

The  distribution  on  the  right  hand  side  of  (4.9)  is  bounded,  continuous  in  u 
and  v,  and  absolutely  continuous  in  x  with  respect  to  Lebesgue  measure.  Thus,  we 
may  substitute  (4.9)  into  (4.5)  ,  pass  to  the  weak  limit  on  n,  and  differentiate 
with  respect  to  x  to  obtain  the  density  f ( • ; t)  of  e(t); 

f(x;t)  =  /Jft(x-[(1 -l)u  +  |v])dF(u,v)  ,  (4.10) 

where  f  is  the  density  of  BT(t) .  This  enables  us  to  calculate 

E(e2(t))  =  /JJx2ft(x-[(l  -l)u  +  |v])dxdF(u,v) 

-  ///(x+(l  -|)u  ♦  |v)2ft(x)dxdF(u,v) 


IV. 5 


=  E[B2(t)]  ♦  (1  -  £)2  E(e2)  +  2^(1  -y)E(ee*)  +  (£)2E(e*2)  (4.11) 

2  t 

with  E[BT(t)]  =  t  ( 1  -  j|r)  .  Finally,  substituting  (1 .11)  into  the  definition  for 
MSE(TA),  we  perform  the  integrations  over  time  to  obtain 

MSE(TA)  =  j  CE(e2)  +  E(ee*)  +  E(e*2))  +  £  .  (4.12) 

For  DM  we  see  that  e*  A  e  +  I,  where  I  is  N(0,T)  independent  of  e.  Thus 
for  DDM  with  c  =  °°, 

MSE(TA)  =  yCE(e2)  ♦  E[e(e+I)]  +  E[(e+I)2]]  +  £  =  E(e2)  + 

which  is  Equation  (11)  in  Masry  and  Cambanis  [14) 

Using  Subsections  A.l  and  A. 2,  where  the  densities  of  e  and  e*  as  well  as 
their  joint  density  were  derived,  it  is  a  straightforward  task  to  calculate 
MSE(SP)  *  E(e2),  MSE(PR)  =  E(e*2)  and  E(ee*) .  The  simplest  calculation  is  for 
MSE(PR) .  By  definition, 

MSE(PR)  *  x2f*(x) dx  .  (4.13) 

From  (4.3  and  4)  we  have 

MSE(SP)  =  u2f(u)du  =  /fco/Too  u2f(u,v)dvdu 

=  MSE(PR)+A2+4AC/7J^A  +  f*(u)<DT(v-u)dvdu 

=  MSE(PR)+A2-4AQ2$(^H)  -  l]uf*(u)du  ,  (4.14) 

u  j'S 

where  4>  is  the  cumulative  standard  normal  distribution  function.  From  (4.3)  we 
have 

E(ee*)  -  £X.  uv  f(u,v)dvdu 

=  MSE(PR)+A2*2A{£J^A  +  /^rAA}(u+v)f*(u)4>T(v-u)dvdu 

*  MSE(SP)-4A/^<K^i)f*(u)du  . 

Substituting  (4.14)  and  (4.15)  in  (4.12)  yields 


(4.15) 


IV. 6 


2 

MSE(TA)  =  MSE(  PR)  +  +  J  +  2A[/!! +  J-J!cA3(u  +  j)f*Cu)4>T(v-u)dvdu 

-  MSE( PR)  +  ^  -  X  *  2u[2$(^H.).l])f*Cu)du.  (4.16) 

B.  Results  and  Discussion 

Using  (4.13-16)  we  compute  MSE(SP)  and  KEE(TA).  Due  to  the  stationarity  of 

the  increments  of  the  Wiener  input  and  the  inclusion  of  A  in  the  (new)  DDM  over¬ 
ly 

load  loop,  MSE(SP)/T  and  MSE(TA)/T  are  both  functions  of  A=AQT  and  c,  but 

h 

MSE(AQT  ,c)/T  does  not  depend  on  T.  Thus,  it  suffices  to  carry  out  the  compu¬ 
tations  for  T=1  only.  The  values  obtained  for  the  MSEs  upon  solving  (4.2)  by 
Galerkin's  Method  (cf.  the  Appendix  regarding  the  use  of  Projection  Methods  and 
the  Method  of  Reduction)  converged  to  four  significant  figures  when  N=5 . 

In  Figures  4  and  5  we  plot  MSE(SP)/T  and  MSE(TA)/T,  respectively,  as  func- 

k  k  k 

tions  of  c  for  A  *  .75T  ,  .9T  and  1.05T  .  Masry  and  Cambanis  [14]  found  that 

for  DM  of  Wiener  input,  min^MSE(SP)  =  .585T  and  min^MSE(TA)  =  1.085T,  with 

k 

AQpt  =  1.05T  for  both.  Our  calculations  with  c  large  agree  with  theirs  to 

about  three  significant  figures.  For  DDM  of  Wiener  input,  MSE(SP)  attains  its 

k 

minimum  of  .4844T  for  (A,c)Qpt  *  (.941T  ,  1.17),  while  MSE(TA)  takes  its  mini- 

k 

mum  value  of  .6698T  for  (A,c)Qpt  =  (.909T  ,  .296).  These  values  represent  17.1% 
and  35.4%  reductions,  respectively,  from  DM.  Note  that  the  conjecture  made  at 
the  close  of  Section  III  has,  in  this  instance,  been  justified.  With  the  same 
rationale,  we  expect  that  the  improvement  obtained  by  using  DDM  instead  of  DM 
for  a  (first-order)  Gauss-Markov  input  will  be  greater  than  that  obtained  for 
an  i.i.d.  input  but  less  than  that  obtained  here. 

-k 

Finally,  in  Figure  6  we  plot  MSE(TA)/T  as  a  function  of  Aq=AT  for  c=0, 

.3,  .6,  and  5.  Once  more  we  find  that  for  a  good  choice  of  c,  the  improvement 
of  DDM  over  DM  is  nearly  uniform  in  A. 


V.  DDM  OF  STATIONARY  FIRST-ORDER  GAUSS -MARKOV  INPUT 


The  analysis  of  the  response  of  DDM  to  stationary  first-order  Gauss-Markov 
input  parallels  that  of  Slepian  [15]  for  DM  of  stationary  Gaussian  inputs  that 
have  rational  spectral  density.  As  in  our  study  of  DDM  of  i.i.d.  input,  we  de¬ 
rive  the  limiting  joint  distribution  of  input  and  output.  This  is  done  by  sol¬ 
ving  the  system  of  integral  equations  obtained  by  conditioning  on  the  preceding 
output  value.  We  then  compute  the  asymptotic  sample  point  and  time-averaged 
mean-square-errors . 

The  input  X^,  t20,  is  taken  to  be  the  stationary  first-order  Gauss-Markov 

process  that  has  mean  zero  and  covariance  function  R(s)  =  ^(^txt+s)  =  e~^sL 

A  -T 

Throughout,  we  let  p  =  e  . 


A.  Analysis 

1 .  The  asymptotic  distribution  of  C^n»xnll) 

We  first  show  that  the  limiting  joint  distributions  of  (Yn=iA,Xn+1) , 
i=0,±l ,±2, . . . ,  exist  weakly  and  have  densities  which  satisfy  a  (CKE)  system  of 
integral  equations  (5.4),  which  is  solved  by  Galerkin's  Method  (see  the  Appendix) 
Let  F^n^(x)  =  P(Yn=iA,Xn+^x)  .  By  conditioning  backwards  we  obtain 

Fin,M  *  P0fn.l=(i-1)4’V1>Vl£!t)  * 


-  P(Vr(i-I)4-Vl>(i'UC)A’Xn.l51t) 

+  P(Ynl=(i-l)A,(i-l-c)A5Xn+1<(i-l+c)A,Xn>(i-l)A,Xn+1Sx) 

♦  P(Ynl=(i+l)A,Xn+1<(i+l-c)A,Xn+1Sx) 

+  PCYn_r(i+1)A’(i+1'c)AiXn+lS(i+1+c)A>Xn-(i+1)A'Xn+l-x)  • 

All  terms  on  the  right  hand  side  of  (5.1)  are  of  the  following  form: 

P(Yn-l“kA*VA'Xn+1tB'Xn+lSx^  P«V  *  *  *  •  VeV 


(5.1) 


/-oo  1B(xn*l)^Ek  VV^i+l(V”*,xn+l)dV,dxn  dxn+l  ’ 


V.2 


where  =  g~_j({kA})  >gn_1(xo*  *  •  *  »xn)  =  Yn_i »  and  ^n+l  is  the  roultivariate  density 
of  (Xp, . . .  »Xn+j)  •  Thus  F^  has  density  f|n^ .  Returning  to  (5.1),  we  have 


F-n)U)  =  /Tco  f^^C^PCX^^Ci-i+OA.X^xlYn  i  =  (i-l)A,Xn*u)du 

+  iIi-l)A  f^1)(u)P((i-l-c)A<Xn+1<(i-l+c)A,Xn+1<x|Yii_1  =  (i-l)A,Xn=u)du 

+  roof^;1)(u)p(Xn+l<(i+1-c)A-Xn+l-xlYn-r(i+1)A'Xn=u)du  C5*2) 

+  J.(i+1)M+i1)(u)P((i+1‘c)A-Xn+lS(i+1+c)A’Xn+l-xlYn-r(i+1)A'Xn=u)du' 

Because  fjn^  <  =  4s *  the  standard  normal  density,  f£n^  (x)  £  (2n)~^  foT 

all  n  and  x.  In  Subsection  II. A  it  was  shown  that  =>  F^ .  Thus,  the  following 

lemma  implies  that  has  density  f^. 

Lemma  3.  Suppose  for  all  n,  Fn  has  density  f^  such  that  f^  £  M  <  «  a.e.  (Leb) . 
If  Fn  =>  F,  then  F  has  density  f  and  f  £  M  a.e.  (Leb). 

Proof :  Let  A  be  any  cpen  set.  Then  for  all  n,  F^(A)  £  M  Leb  (A),  and  there¬ 
fore  F(A)  <  M  Leb  (A).  Hence  F  is  absolutely  continuous  with  respect  to  Lebesgue 
measure,  and  its  density  f  £  M  a.e.  (Leb).  □ 

Because  Y  ,  =  g  ,(X„,...,X  )  and  X  is  Markovian,  we  have  (a.e) 
n-1  &n-l  0’  ’  n  n 

P(VliA'Vla|*»=“'Vl-l4)-P(VlfA'VialSns“)=^  lA(v)»T(v|u)du  ,  (5.3) 

where  <f>T  denotes  the  conditional  density  of  Xt+T  given  Xt>  This  is  a  bounded  (by 
one)  and  continuous  function  of  u,  and  an  absolutely  continuous  function  of  x. 

Hence,  substituting  (5.3)  into  (5.2),  passing  to  the  limit  in  n,  and  differentia¬ 
ting  with  respect  to  x,  we  obtain  the  following  system  of  integral  equations  for 


the  asymptotic  joint  densities  {f  }  a.e.;  for  i=0,  ±  1,  ±2,..., 


f.OO  .  q.jCu)  *  Ib.OOJ^.d^OO 


(5.4) 


'c.00^.  Wu)  *  fj,!  (u)}4>T(x|u)du, 


V.  3 


where 

\  =  ((i-l+c)A,°°) ,  Bi  =  [  (i-l-c)A,  (i-l+c)A]  , 

C±  =  (-00,  (i+l-c)A)  ,  Di  =  [ (i+l-c)A, (i+l+c)A]  . 

Note  that  setting  c  =  °°  reduces  (5.4)  to  Eq.  (23)  in  Slepian  [15].  Of  course, 
in  addition  to  (5.4)  we  have  the  boundary  condition 

I  f±  =  4>  a.e.  (5.4.1) 

i 

Let  f  =  (...,f  ,fg,f  ,...)^.  Since  from  (5.4.1)  we  have  that 

_1  "  1  i 

fet^C^C^  ))•  In  Subsection  C  of  the  Appendix  we  show  that  the  system  of  inte¬ 
gral  operators  A  given  by  the  right  hand  side  of  (5.4)  is  bounded  and  linear  on 

-£-2 ( L2 (^  *))»  and  that  f  is  the  unique  solution  to  g  =  Ag  subject  to  jg.  =  <J>.  f 

i  1 

is  found  by  Galerkin's  Method  (see  the  Appendix). 

2 .  The  (mean  square)  sample  point  error. 

Having  found  the  limiting  distribution  of  (Yn>xn+1)>  we  derive  in  C5.5)  the 
limiting  distribution  of  (Yn>xn)*  which  enables  us  to  calculate  the  -*>y'iptotic 
mean  square  sample  point  error  (5.6). 

Let  G^n^(x)  =  P(Yn=iA,XnSx) .  Conditioning  backwards  yields 

c[n,(x)  =  duff^OOmx^jCA.Ix^u)  *  l(u>(i-l) A)P(Xn+1«Bi |Xn*u) ] 

*  fuI1)cP(1WCilVu)  *  Hu^ti-DAJPtX^jCDjx^u)]} 

Since  G^n^  =>  G^  that  has  density  g^,  we  have 

?i(x)  =  f._1(x)[P(Xn+1£Ai|Xn=x)  *  l(x>(i-l)A)P(Xn+1eB.|Xn=x)] 

♦  fi+l(x)[P(Xn+i€Ci|Xn*x)  *  l(x<(i4l)A)P(Xn+1£D.|Xn=x)] 

=  {f (x)  [  JA  +1  (x>  (  i-1) A)  JB  ] 

+  fi+l(x^C^C  +1(x:s(i+l)A)/  ])<i>T(y|x)dy  , 

i  i 


(5.5) 


V.4 


and  just  as  in  (3.7)  and  (3.9), 


MSE(SP)  =  1-2*1  i/^  x  gi(x)dx  +  *2l  i2p. 


3.  The  time-averaged  mean  square  error. 

We  calculate  the  time-averaged  mean  square  error 

MSE(TA)  £  i/J  E[e2(t) ]dt  , 

where  e(t)  is  the  limit  in  distribution  of  e  (t)  =  X  _  -Y  in  much  the  same 
^  J  n^  1  nT+t  nT’ 

way  as  Slepian  [  ).  Its  expression  is  given  in  (5.10). 

For  te(0,T),  let  F^n^(x;t)  =  P(Yn=iA,XnT+t.Sx)  .  We  show  first  that  F^(*;t) 
has  weak  limit  F^(*;t),  and  that  all  of  these  distributions  have  densities. 

The  same  steps  used  in  the  derivation  of  (5.2)  yield 

-  Jl  f^;1)(u)P(Xn,1cA.,XiiT,tSx|Yii.1=(i-l)fi,  Xn=u)du 

*  fl-;1)WPtVl5Bi-XnT.tMlYn-r(i-”A'Vu’du 


(S.6) 


Recalling  that  ^  =  8n  j(Xq, . . .,Xn),  and  X^,  t^O,  is  first-order  Markovian, 
we  have  (a.e.) 

P^Xn+leE,XnT+t$x^Yn-l=k^’Xn=u^  =  P<WE’ Wx  lXn=u> 

=  /E/^oo  <t)T_t  (w  I  v)  <j>t  ( v  |  u)  dvdw . 

Since  F?n^  =>  F^  that  has  density  f^,  from  (5.7  and  8)  we  get 

*!<*!*>  '  *  [^C.  * 
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4>T.t(w|x)<})t(x|u)dwdu, 


(5.7) 


(5.8) 


(5.9) 


and  so  similar  to  (5.6)  we  have 


MSE(TA)  =  I  l  jj£.  (x-iA)2  f.(x;t)dxdt 

=  1+A2  y  i2pi  -  -y-  I  i/J/Toe  xfi(x;t)dxdt.  ( 

B.  Numerical  Results  and  Discussion 

The  numerical  analysis  (cf.  the  Appendix  regarding  the  use  of  the  Method  of 
Reduction)  is  carried  through  for  T=.51,  .36  and  .22  (p=.6,  .7  and  .8).  The 
values  for  the  MSEs  converged  to  five,  four,  and  three  significant  figures, 
respectively,  when  N=10,  13  and  15  respectively.  In  every  case,  the  value  of 
I  ceased  to  have  an  effect  when  (I-l)A  >  3.5,  i.e.  when  the  largest  output 
level  exceeded  3.5. 

In  Figures  7  and  8  we  graph  MSE(SP)  and  MSE(TA),  respectively,  as  functions 
of  c  for  T=.51  with  A=  .45,  .55,  and  .65,  and  for  T=.36  with  A=.45,  .525,  and 
.6.  For  large  c,  our  calculations  for  MSE(TA)  appear  in  good  agreement  with 
Slepian's  [15].  The  two  sets  of  plots  in  each  figure  are  very  similar  both  to 
each  other  and  to  their  corresponding  Wiener  input  plots.  This  is  encouraging 
in  that  it  points  to  a  certain  uniformity  in  the  performance  of  DDM  for  posi¬ 
tively  correlated  signals. 

In  Tables  II  and  III  we  display  the  minimum  values  of  MSE(SP)  and  MSE(TA), 
respectively,  with  the  (optimal)  parameter  setting(s)  that  yield  these  minima, 
for  both  DM  and  DDM.  We  find  that  MSE(SP)  is  reduced  about  17%,  while  MSE(TA) 
is  reduced  about  37%,  by  using  optimal  DDM  instead  of  optimal  DM.  In  every  case 
the  optimal  value  of  A  is  smaller  for  DDM  than  for  DM.  Note  that  these  results 
are  quite  similar  to  those  found  for  a  Wiener  input. 


VI.  CONLCUDING  DISCUSSION 


A.  The  effects  of  perfect  integration  and  step-function  reconstruction  on 


mean-square-error  and  input-output  synchronization. 

Recall  that  when  c=  00  ,  =  YD^,  while  when  c=0,  y^M  =  Y^, ,  so  that  the 

n  n  n  n+1 

(c=0)-DDM  output  is  just  a  T-shift  of  the  DM  output  for  any  choice  of  reconstruc- 


tion  Y^  based  on  {Y  }.  This  gives  us 
-  t  n 


MSE(TA;(c=0)-DDM)  =  "lim*  =jr  jJ  EUXnT+t-Y^t)2]dt 


=  "lim"  T  JQ  E[(xnT+t'Y(n+l)T+t)  ]dt 


=  MSE(TA;  T-shifted  DM), 

where  "lim"  a  =  lim  h(a  +a  ,)  since  Y  has  period  two.  Hence,  from  Figures 
n  n  n  n  n+l  n  .0 

5  and  8  we  see  that  for  DM  with  perfect  integration  and  step  function  reconstruc¬ 
tion  Yt=YnT  tc[nT,(n+l)T) ,  about  one  third  of  MSE(TA)  is  due  to  a  lag  in  the 
tracking,  rather  than  a  distortion  of  the  shape,  of  the  signal.  This  has  an 
important  implication  regarding  the  use  of  the  (time-averaged)  mean  square  error 
criterion.  When  one  is  mainly  interested  in  preserving  the  shape  of  the  input 
and  associates  no  loss  to  small  time  lags,  such  as  in  speech  or  image  transmission, 
a  more  appropriate  measure  of  system  performance  may  be 

MSE(TA; I-shifted)  =  min  "lim"  i  /J  E[ CxnT+t'YnT+t+s)2]dt’ 


sel  n 


where  I  is  an  interval  that  corresponds  to  the  time  shifts  deemed  acceptable. 
In  Figures  9  and  10  we  graph  MSE(SP)  and  MSE(TA) ,  respectively  for  DM  and 


T-shifted  DM  (i.e.  DDM  with  c=°°  and  c=0)  with  perfect  integration  and,  in  the 

time-averaged  case,  step-function  reconstruction,  of  stationary  first-order 

Gauss-Markov  input.  We  see  that  while  MSE(SP;  T-shifted  DM)  <  MSE(SP;DM)  only 

when  A  is  small  (so  small  that  Y^M  cannot  "keep  up"  with  X^) ,  MSE(TA;  T-shifted  DM) 

DM 

<  MSE(TA;DM)  uniformly  in  A,  i.e.  Y  lags  behind  Xt  equally  for  all  A.  Thus, 


VI. 2 


the  lag  in  the  tracking  of  Xt  by  Y  here  is  mainly  a  result  of  the  use  of  the 
step-function  reconstruction. 

It  is  clear  that  MSE(SP)  does  not  depend  on  the  reconstruction  and  measures 
only  what  may  be  called  the  "encoding-decoding  error"  of  the  system,  while 
MSE(TA)  measures  both  the  encoding-decoding  error  and  the  "reconstruction  error." 
In  general,  the  reconstruction  cannot  compensate  for  deficiencies  in  encoding¬ 
decoding.  Specifically,  given  different  "modulations"  Y  ,  Z n  of  Xn>  n=0,l,2,..., 
and  using  the  same  reconstruction  for  both  Y^  and  Z^,  if  MSEyCSP)  <  MSEZ(SP), 
we  expect  that  MSEy(TA)  <  MSEZ(TA).  However,  one  expects  that,  in  most  cases, 
MSE(TA) -MSE(SP) ,  though  positive,  can  be  made  small  by  using  more  sophisticated 
reconstructions,  which  have  higher  complexity  and  require  greater  memory.  For 

example,  for  a  Wiener  input  W^,  t>0  ,  suppose  we  are  given  the  perfect  modula- 

SF 

tion  Yn  =  WnT,  n=0,l,2,...,  the  step-function  reconstruction  YnT+t  =  Y^ ,  and 
the  linearly  interpolated  reconstruction  Y^*+t  *  (1-  ^)Yn  +  ^  Yn+j »  f°r  ter0,T). 
Although  in  this  case  MSE(SP)  =  0,  it  is  easily  verified  that  MSEfTA:SF)  =  T/2 
and  MSE(TA:LI)  =  T/6.  When  the  performance  of  the  reconstruction  is  of  secondary 
interest  to  that  of  the  modulator,  MSE(SP)  is  the  more  appropriate  criterion. 

B.  Unification  of  results. 

The  inputs  analyzed  in  Sections  III-V  have  samples  X^,  n=0,l,2,...,  that 
may  be  viewed  as  generated  by  the  first-order  autoregressive  sequence  Xn+1  = 
pXn  +  a  In+1>  where  0sp<l,  0<a<l ,  and  {i^}  are  i.i.d.  N(0,1).  In  Section  III 
we  have  p=0,  a=l;  in  Section  IV  we  have  p=l,  a=l;  in  Section  V  we  have  0<p<l , 

2  *5 

a  =  (1-p  )  .  In  Figure  11  we  plot,  for  DM  and  DDM  with  perfect  integration, 

-2  -2 

a  min^  MSE(sp)  anci  a  c^flSE(SP),  respectively,  as  functions  of  p. 

2 

Dividing  by  a  "standardizes"  the  MSE  with  respect  to  the  increment  power.  The 
optimal  "standardized"  performance  of  both  DM  and  DDM  is  seen  to  vary  only 
slightly  with  p,  and  the  improvement  attainable  by  using  DDM  instead  of  r*1  to 
be  nearly  uniform  in  p. 


VI. 3 


Amstein  [1]  has  analyzed  DM  with  matched  integration  when  ot=l,  and  his 

plot  of  min^MSE(SP)  as  a  function  of  p  in  Figure  4.2  is  included  inFigure  11. 

We  see  that  for  DM  of  first  order  autoregressive  inputs,  the  loss  incurred  by 

using  the  perfect  instead  of  the  matched  integrator  is  less  than  10%  when  p  >  .8, 

and  vanishes  as  p  -*■  1 .  In  addition,  we  note  the  following:  for  DDM  with  matched 

integration,  when  p  =  0,  a  =  1,  min^  MSE(SP)  =  .3634  (at  (A,c)  =  (.798,°°) ;  cf. 

Fine  [5]),  and  when  p=l,  a=l,  min^  MSE(SP)  =  .4844  (cf.  Subsection  IV. C). 

-2 

For  DDM  with  matched  integration,  we  expect  that  a  plot  of  a  min^  c)  MSE(SP) 
versus  p  will  resemble  Amstein's  [l;Figure  4.2]  matched  integration  DM  curve  and 
have  .3634  and  .4844  (instead  of  .585)  as  limits  at  zero  and  one,  respectively. 

A  conjecture  for  this  curve  is  also  included  in  Figure  11. 


In  Figure  12  we  graph  A^^/ot  (with  respect  to  MSE(SP)  as  a  function  of  p  for 

DM  and  DDM  with  perfect  integration.  Note  that  A^^/a  varies  only  slightly 
2  h 

with  p  =  (1-a  )  ,  and  thus  also  with  a;  it  is  encouraging  that  the  optimal  A  has 


a  linear  relation  with  the  increment  scale.  The  reduction  in  A  /a  for  DDM 

opt 

from  DM  increases  monotonically  from  3%  to  10%  as  p  goes  from  zero  to  one. 


Comparing  the  DM  curve  with  Amstein' s  [l;Figure  4.1]  plot  of  A  ^  versus  p  for 
DM  with  matched  integration  when  ot=l,  included  in  Figure  12,  reveals  that  for 
0«P<1  the  matched  integrator  also  permits  the  use  of  slightly  smaller  A.  This 
is  because  with  the  matched  integrator  the  random  variables  to  be  (binary) 


quantized,  i.e.  the  predictive  errors,  are  in  general  smaller,  and  because  the 


matched  integrator  itself  compensates  for  the  more  common  type  of  slope  over¬ 
loads,  those  during  which  |Xt|  -*■  0.  Using  the  matched  integrator  in  DDM  should 
produce  a  similar  reduction. 

In  Figure  13  we  graph  c  ^  (with  respect  to  MSE(SP))  as  a  function  of  p.  As 
expected,  the  more  positive  the  correlation  of  the  samples,  the  more  useful  the 
new  DDM  overload  loop,  and  hence  the  smaller  the  optimal  value  of  c.  It  is  in¬ 


teresting  that  the  plot  is  highly  linear. 


APPENDIX.  TOE  SOLUTION  OF  INFINITE  LINEAR  SYSTEMS 


In  each  of  our  analyses,  we  encounter  the  problem  of  solving  an  infinite 
linear  system.  In  our  first  analysis,  we  show  that  the  joint  distributions 
{F?(»}}  may  be  expressed  in  terms  of  the  constants  {p^,q^l  and  the  input  dis¬ 
tribution  H(*)  (cf.  (3.1)).  We  then  derive  an  infinite  system  of  homogeneous 
linear  equations  in  these  constants  (cf.  (3.2-5))  and  a  normalizing  "boundary" 
condition  (cf.  (3.6)).  This  infinite  linear  system  is  solved  by  the  Method  of 
Reduction.  This  involves  setting  p^  =  0  =  q^  for  all  i  >  I,  solving  the 
resulting  finite  system,  and  taking  as  the  solution  for  the  infinite  system  the 
limit  of  the  finite  "approximating"  solution  as  I  t  »  .  (See  Kantorovich  and 
Krylov  [11]  for  more  on  the  Method  of  Reduction.)  In  the  first  subsection 
below  we  show  why  this  may  be  done. 

In  our  last  two  analyses,  the  desired  (joint)  density  is  shown  to  satisfy 
a  (system  of)  linear  integral  operator  equation(s)  of  the  second  kind  (cf. 

(4.2)  and  (5.4))  as  well  as  a  normalizing  boundary  equation  (cf.  (4.2.1)  and 
(5.4.1)),  and  to  belong  to  a  Hilbert  space.  Henceforth  we  let  f  denote  the 
desired  (joint)  density; 

g  =  Ag  (A. 1) 

represent  the  (system  of)  integral  operator  equation(s)  ((4.2)  or  (5.4))  for 
which,  subject  to  a  given  normalization,  f  is  the  unique  solution;  H  be  the 
Hilbert  space  to  which  f  belongs;  and  (en)  be  a  complete  and  orthonormal 
system  in  H.  Since  feH,  it  has  Fourier  expansion 

f(0  =  Ifnen(°  ’  (A*2) 

where  the  Fourier  coefficients  are  given  by 


A.  2 


Substituting  (A. 2)  into  (A.l)  and  taking  inner  products  (in  H)  on  both  sides 
of  the  resulting  equation  yields  an  infinite  system  of  linear  equations  in  the 
{fn},  which  is  solved  by  the  Method  of  Reduction.  This  procedure  is  known  as 
Galerkin's  Method,  one  of  many  so-called  Projection  Methods  for  the  solution  of 
(integral)  operator  equations.  (For  more  on  these  methods  see  Krasnoselskii 
et.al.  [13]  and  Zabreyko  et.al  [16].)  In  Subsections  B  and  C  below  we  show  that 
the  (system  of)  integral  operator(s)  A  is  bounded  on  H.  It  then  follows  (cf. 

Gerr  [7]  and  Gerr  and  Cambanis  [6])  that  Galerkin's  Method  may  be  used. 

A.  Majorant  Systems  and  the  Method  of  Reduction 

We  first  note  several  properties  of  the  infinite  system  of  linear  equations 
obtained  in  the  analysis  of  DDM  of  i.i.d.  input.  It  is  composed  of  an  infinite 
system  of  homogeneous  linear  equations,  given  by  (3.2  and  3),  in  the  unknowns 
{p. ,q. ,  supplemented  by  the  non-homogeneous  normalization  (3.6).  Thus 
it  has  the  following  form: 

Xj  =  ][a„x_. ,  i  =  0,  ±1,  ±2,...,  (A. 3.1) 

=  b  ,  (A. 3. 2) 

i 

where  b  =  J\(p^+q^)  =  Using  (3.5)  appropriately  in  (3.2  and  3)  results  in 

0  s  a.  .  <  1  for  all  i,j,  with  a. .  +  0  for  at  most  four  values  of  j  for  all  i.  As 
ij  lj 

a  shorthand  we  write  x  *  Ax  for  (A. 3.1)  and  define 

p  =  (...,  p.1,q.1,p0»q0,p1,q1,-*-)T  . 

Theorem  A.l.  If  z  c  2^  satisfies  (A. 3.1),  then  z  =  (1.5)”*  (£.z^)p.  Thus 
(A.l)  has  unique  solution  in  2^. 

Proof:  Let  K  denote  the  positive  cone  in  2j  ,  i.e.  K  =  {x  e  2^;  x^  £  0  all  i}, 
and  let  y  In  Section  II. C.  it  was  shown  that  =>  F^  from  any  initial 

state  and  so,  in  particular,  pfn^  +  pA  and  q|n^  +  qA .  Thus  Any  ■+  Cl .  5)  ~ |  y||  p 


pointwise.  As  z  c  £j  may  be  written  Z  -  z^-z^,  where  z^,  z^  e  K,  we  have 

that  A11?  (1.5)”1(^zi)f5  pointwise,  with  lJ.z^|  *  ||  z||  <  00  .  Since  AZ  =  z, 

a"z  =  Z  for  all  n,  hence  z  =  (1.5)"*(J^z.)p  .  □ 

This  implies  that  there  is  a  single  linear  dependency  in  A.  If  we  sum  the 
equations  (A. 3.1)  over  i  we  obtain  the  trivial  identity  J\x^  =  .  Thus  the 

system 

»i*£jaij  V  1-±1-±2 .  (A-4-1) 

Iixi  =  1  ,  (A.4.2) 

has  p  as  its  unique  solution.  It  now  follows  that  the  system 

Xi  =  ^jaijXj ’  1  =  Al*  (A. 5.1) 

x0  =  1  , 

has  as  its  unique  solution  q  =  p/pQ  (having  identified  xQ  with  pQ  >  0) .  In 
Theorem  A. 2  below  we  show  that  q  may  be  found  by  the  Method  of  Reduction  so  that 
finally  p  =  pQc[  =  1.5||  ?||  _1q  . 

Theorem  A. 2.  The  system  (A. 5)  can  be  solved  by  the  Method  of  Reduction. 


Proof:  In  general,  an  infinite  system  of  linear  equations  in  the  infinite  set 
00 

of  unknowns  ix.}.  may  be  written  as 

l  1=-°° 

X1  “  *  b0  •  i=0,  JJ,  ±2 .  (A.6) 

The  system  of  equations 


x.  =  T.a. .x. .  +  b. 

i  LJ  11  lj  3 


i=0,  ±1,  ±2,. ... 


is  "Marjorant  for  the  system  (A. 6)”  when  for  all  i  and  j,  |a^|  ^  A^  and 

|b.|  s  B..  The  system  (A. 5)  is  its  own  majorant,  and  by  Theorem  A.l,  has  unique 
solution  q  e  K,  the  positive  Jij  cone.  The  result  then  follows  from  Theorems  I-IV 
on  pages  20-26  in  Kantorovich  and  Krylov  [11].  □ 


We  prove  below  that  the  integral  operator  A  given  by  the  right  hand  side 
of  (4.2)  is  bounded  on  L2(Leb).  To  prove  this  it  suffices  to  show  that  the 
integral  operator  B  defined  by 

[Bf ]  (x)  =  l(x>(c-l)A)/^oo4>T(x+A-u)f (u) du  (A.7) 

is  bounded  linear  operator  on  l^CLeb),  the  other  terms  being  treated  similar¬ 
ly. 

Clearly,  B  is  linear.  From  (A.7)  we  have 

II  Bfli2  «  /TcoM2 

-  rjToo/!lo^TCx+A-u)*T(*+A-v)  |f(u)f(v)  Idudvdx 

s  rjj2T^f2w^ 

-  O2  *  II  *11  2  •  (A.  8) 

Hence  B  is  bounded,  with  ||  B||  £  1. 

It  is  easily  shown  that  B  is  not  compact  on  L2(Leb).  Similar  to  (A. 8)  we 

have  ||  Bf  ||  2  =  -cA)<f>2T(u-v)f  (u)f  (v)dudv.  Putting  fn(x)  = 

w 

l(n  <  x  s  n  ♦  1) ,  we  see  that  although  f^  -*■  0  in  L2(Leb), 

II  Bf„ ||  2  =  -  cA)<^2T(u-v)dudv 

n  /o^0^2T(u'V^dudv  >  °  * 

and  so  Bf  -AO  in  L_(Leb). 
n  2 

As  a  final  consequence  of  this  subsection  we  have  that  the  integral  opera¬ 


tor  D  defined  by 


[Df ] (x)  =  (x-u)f(u)du 


is  bounded,  with  ||  D||  si,  but  not  compact,  on  L,(Leb) 


C.  DDM  of  Stationary  First-order  Gauss-Markov  Input 


We  prove  below  that  the  system  of  integral  operators  A  given  by  the  right 
hand  side  of  (5.4)  is  a  bounded  linear  operator  on  H  =  ■  Note  that 

in  this  case,  since  every  g  e  H  is  integrable,  the  fact  that  f  e  H  is  the 
unique  integrable  solution  to  (5.4)  implies  that  1  is  a  simple  eigenvalue  of  A. 

Let  g  e  H.  Using  Mehler's  Formula 

00 

4>t(x|u)  =  4>(x)  l  pnH  (x)Hn(u)  , 
n=0 

where  Hn(*)  is  the  Hermite  polynomial  of  order  n,  from  (5.4)  we  have 

IIWlS  •ijj(X|)ill2L2(*-1) 

00 

*  8  I  ///{|gi(u)g.(v)  |  +  |gi+1(u)gi  l(v)  |}<J>T(x|u)4)T(x|v)4)"1(x)dudvdx 


=  8  I  Ip  "//{ig. (u)g. (v)|+|g. +1(u)g,  . (v) | }H  (u)H  (v)dudv 
i=-«>  n=0  1  1  1  n 


“  8.  I  I  P2n[<|giU  H  *  >2  +  <|gi+1l,  H  ♦  ><[gi_1l  ,  H  4>  >] 
i=-°°  n=0 


5  16||?||2  . 

where  the  inner  product  <•»•>  is  in  L2 C4>  ).  Thus  A  is  a  bounded  linear  opera¬ 

tor  on  (L_  (4>-1) )  . 


REFERENCES 


[1]  ARNSTEIN,  D.S.,  "Predictive  source  coding  of  correlated  Gaussian  pro¬ 

cesses",  Ph.D.  Dissertation,  Univ.  of  California,  Los  Angeles,  1972. 

[2]  BILLINGSLEY,  P.,  Convergence  of  Probability  Measures.  New  York:  Wiley, 

1968. 

[3]  CHAN,  H.C.  and  ANDERSON,  J.B.,  "Adaptivity  versus  tree-searching  in  DPCM" 

IEEE  Trans.  Commun.,  vol.  C0M-30,  pp.  1254-1259,  1982. 

[4]  DOOB,  J.L.,  "Asymptotic  properties  of  Markoff  transition  probabilities", 

Trans.  Amer.  Math.  Soc.,  vol.  63,  pp.  393-421,  1948. 

[5]  FINE,  T.L.,  "The  response  of  a  particular  non-linear  system  with  feedback 

to  each  of  two  random  processes",  IEEE  Trans.  Inform.  Theory,  vol. 
IT-14,  pp.  255-264,  1968. 

[6]  GERR,  N.L.  and  CAMBANIS,  S.,  "Analysis  of  an  adaptive  differential  pulse- 

code  modulator",  manuscript. 

[7]  GERR,  N.L.,  "Exact  analysis  of  a  delayed  delta  modulator  and  an  adaptive 

differential  pulse-code  modulator".  Tech.  Rep.  #22,  Center  for 
Stochastic  Processes,  Dept,  of  Statistics,  Univ.  of  North  Carolina 
at  Chapel  Hill,  1982. 

[8]  GERSHO,  A.,  "Stochastic  stability  of  delta  modulation".  Bell  Syst.  Tech. 

J.,  vol.  51,  pp.  821-841,  1972. 

[9]  HAYASHI,  A.,  "Stochastic  stability  of  DPCM  for  the  Wiener  process  input", 

Trans.  IECE  Japan,  vol.  J61-A,  pp.  180-181,  1978. 

[10]  JANARDHANAN,  E.,  "Differential  PCM  -ystems",  IEEE  Trans.  Commun.,  vol. 

Com-27,  pp.  82-93,  1979. 

[11]  KANTOROVICH,  L.V.  and  KRYLOV,  V.I.,  Approximate  Methods  of  Higher  Analysis 

New  York:  Interscience,  1958. 

[12]  KIEFFER,  J.C.,  "Stochastic  stability  for  feedback  quantization  schemes", 

IEEE  Trans.  Inform.  Theory,  vol.  IT-28,  pp.  248-254,  1982. 

[13]  KRASNOSEL'SKII ,  M.A.  et.al..  Approximate  Solution  of  Operator  Equations. 

Groningen:  Wolters-Noordhoff ,  1972. 

[14]  MASRY,  E.  and  CAMBANIS,  S.,  "Delta  modulation  of  the  Wiener  process", 

IEEE  Trans.  Commun.,  vol.  COM-23,  pp.  1297-1300,  1975. 

[15]  SLEPIAN,  D.,  "On  delta  modulation".  Bell  Syst.  Tech.  J. ,  vol.  51, 

pp.  2101-2137,  1972. 

[16]  ZABREYKO,  P.P.  et  al..  Integral  Equations:  A  Reference  Text. 

Leyden :Noordhoff,  1975. 


TABLE  I 


Output  Level  Probabilities 


Po 

DDM  w/(A,c)opt 
.383750 

DM  w/A 

opt 

.384036 

Pi 

.246477 

.246382 

p2 

.058091 

.057994 

P3 

.003522 

.003618 

P4 

.000034 

.000038 

Pc 

.00000003 

.00000004 

TABLE  II 


minMSE (SP)  and  Optimal  Parameters  of  DM  and  PPM 
DM  DDM 


f 

minMSE(SP) 

■^opt 

minMSE (SP) 

C^opt 

6 

.357 

.72 

.296 

[. 64, 1.57) 

7 

.287 

.66 

.237 

(.59,1.47) 

8 

.201 

.57 

.166 

(.52,1.37) 

minMSE (TA)  and 

TABLE  III 

Optimal  Parameters  of 

DM  and  DDM 

DM 

f 

minMSE (SP) 

A  „ 
-opt 

minMSE (TA) 

6 

.675 

.57 

.430 

(.56, .42) 

7 

.545 

.57 

.338 

(.54, .37) 

8 

.380 

.53 

Figure  3 
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